Methods and apparatus for leveraging sentiment values in flagging and/or removal of real time workflows

ABSTRACT

Aspects of the disclosure relate to using an apparatus for flagging and removing real time workflows that produce sub-optimal results. Such an apparatus may include an utterance sentiment classifier. The apparatus stores a hierarchy of rules. Each of the rules is associated with one or more rule signals. In response to receiving the one or more utterance signals, the classifier iterates through the hierarchy of rules in sequential order to identify a first rule for which the one or more utterance signals are a superset of the rule&#39;s one or more rule signals. In response to receiving the one or more alternate utterance signals from the signal extractor, the classifier may iterate through the hierarchy of rules in sequential order to identify the first rule in the hierarchy for which the one or more alternate utterance signals are a superset of the first rule&#39;s one or more rule signals.

FIELD OF TECHNOLOGY

Aspects of the disclosure relate to leveraging sentiment values for usein interactive voice response systems.

BACKGROUND OF THE DISCLOSURE

Interactive response systems are computer-based systems that communicatewith users of the system. Exemplary interactive response systems includeinteractive voice response systems configured to communicate with a uservia verbal messages, and chatbots configured to communicate with a uservia text messages. The interactive response systems can receive inputsfrom the user via text, audio and/or gesture (e.g., selection of adisplayed icon).

The interactive response systems receive inputs from a user. After aninput is received, the system generates a response. Generating theresponse typically includes iterating through pre-stored rules toidentify a suitable response based on the received input.

At times, the response generated by the interactive response system maybe misunderstood by, or frustrate, the user. Such suboptimal responsesmay result in user disappointment with an entity hosting the interactiveresponse system, and may result in negative reviews, unfulfilledbusiness opportunities and, possibly, the loss of the user as an entitycustomer. However, as the interactive response system is an automatedcomputer system, user emotion such as frustration or discontent is notidentified by the system and, as such, is not addressed during theuser-system interactions.

At this time, options exist for a customer to rate his interaction withthe interactive response system by selecting a happy or sad smiley faceafter the interaction. This rating system is not useful at least becausea customer rarely selects the happy/sad smiley face after theinteraction and, even when the face is selected, it provides no clues asto which system response triggered the positive/negative review.

It would be desirable, therefore, to provide systems and methods forenabling the interactive voice response system to identify and scoreuser sentiment in real time for each discrete portion of a conversation.The scoring of user sentiment for discrete conversation portions may beused by the interactive response system to select an appropriate systemresponse to each user utterance and to pinpoint suboptimal systemresponses.

Machine learning (“ML”) is used today for a variety of predictivepurposes. ML leverages an algorithm trained using training data to makeits predictions. Using machine learning to predict sentiment of aconversation, however, presents a programming team with multipledifficulties. A large number of words/phrases may be used whenconversing with the interactive response system. Thus, the volume oftraining data required to train the algorithm for recognizing sentimentis necessarily huge. Additionally, because understanding sentiment of acurrent utterance may require knowledge of previous utterances, a largevolume of data must be input into the ML to understand the utterance incontext. Such methods require the ML algorithm to utilize considerableprocessing resources to identify user sentiment and may result indelayed output.

As such, it is further desirable to provide systems and methods thatleverage ML to predict conversation sentiment without necessitatingcomplex training data sets and large data inputs.

It is yet further desirable to provide systems and methods that leverageML to remove or otherwise deactivate workflows that produce sub-optimalresults.

SUMMARY OF THE DISCLOSURE

A method for pre-processing of an utterance prior to feedingutterance-related data to a sequential neural network classifier forconversation sentiment scoring is provided. The utterance may beexpressed, by a user, to an interactive response system during aninteraction between the user and the interactive response system. Themethod may include using a conversation manager to receive a statelessapplication programming interface (“API”) request including theutterance. The conversation manager may also be used to receive previousutterance data and a sequence of labels. Each label may be associatedwith a previous utterance expressed by a user during the interaction.

The method may also include using a natural language processor toprocess the utterance to output an utterance intent, a semantic meaningof the utterance and an utterance parameter. The utterance parameter mayinclude one or more words included in the utterance and may beassociated with the intent.

The method may also use a signal extractor to process the utterance, theutterance intent, the semantic meaning, the utterance parameter, and theprevious utterance data. The signal extractor may generate one or moreutterance signals. The one or more utterance signals may correspond to amost probable intent of the user.

When the one or more utterance signals has been previously generatedduring the interaction and has previously not satisfied the user, thenthe method may include modifying the one or more utterance signals tocorrespond to the next most probable intent of the user.

The method may also use an utterance sentiment classifier for storing ahierarchy of rules. Each rule in the hierarchy of rules may beassociated with one or more rule signals and a label. In response toreceiving the one or more utterance signals from the signal extractor,the method may iterate through the hierarchy of rules in sequentialorder to identify a first rule in the hierarchy for which the one ormore utterance signals are a superset of the first rule's one or morerule signals.

The method may also use a sequential neural network classifier forreceiving a data input including the sequence of labels and a labelassociated with the rule identified by the utterance sentimentclassifier. Preferably, the data input does not include the utterance.The sequential neural network classifier may also be used for processingthe data input using a trained algorithm and for outputting a sentimentscore.

The method may use the conversation manager for identifying a responseto the user utterance based on the utterance intent, the label and thesentiment score and for augmenting the stateless API request to includethe response, the utterance intent, the semantic meaning, the utteranceparameter, the label and the sentiment score. After the augmenting, theconversation manager may be used for transmitting the stateless APIrequest to the interactive response system. The interactive responsesystem may receive the stateless API request and output the response tothe user.

The pre-processing of the utterance by the natural language processor,the signal extractor and the utterance sentiment classifier preferablyreduces the data input to the label and the sequence of labels, therebyincreasing a speed at which the sequential neural network classifierreturns the sentiment score and decreasing resources consumed by thesequential neural network classifier when processing the data input.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and advantages of the invention will be apparent uponconsideration of the following detailed description, taken inconjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows illustrative apparatus and methods in accordance with theinvention;

FIG. 2 shows an illustrative diagram in accordance with the invention;

FIG. 3 shows an illustrative diagram in accordance with the invention;

FIG. 4A shows an illustrative flow diagram in accordance with theinvention;

FIG. 4B shows an illustrative flow diagram in accordance with theinvention;

FIG. 5A shows an illustrative flow diagram in accordance with theinvention;

FIG. 5B shows an illustrative flow diagram in accordance with theinvention;

FIG. 6 shows an illustrative flow diagram in accordance with theinvention;

FIG. 7 shows an illustrative flow diagram in accordance with theinvention;

FIG. 8 shows illustrative block diagram of apparatus in accordance withthe invention;

FIG. 9 shows illustrative apparatus that may be configured in accordancewith the invention;

FIG. 10 further granularity of the apparatus shown in FIG. 1 , inaccordance with the invention; and

FIG. 11A shows another illustrative flow diagram in accordance with theinvention;

and

FIG. 11B shows an extension of the illustrative flow diagram in FIG. 11Ain accordance with the invention.

DETAILED DESCRIPTION OF THE DISCLOSURE

Apparatus and methods for conversation sentiment scoring is provided.The apparatus and methods may include a conversation manager configuredto receive requests, each request including an utterance. The utterancemay be an utterance expressed by a user during an interaction betweenthe user and an interactive response system. The interactive responsesystem may be an interactive voice response system configured to outputverbal messages to the user. The interactive voice response system mayalso be configured to transmit information to a user's computing device(referred to alternately herein as a ‘user device’). The interactiveresponse system may be a chatbot configured to output to the user textmessages, verbal messages, selectable icons and/or any other suitabledata.

When the interactive response system is an interactive voice responsesystem, the utterance may be a verbal message and/or one or more keypadselections. When the interactive response system is a chatbot, theutterance may be a text message and/or a selection of one or moreselectable icons displayed, to the user, on a graphical user interfaceof the user's device. Exemplary user devices include a laptop,smartphone, smartwatch, or any other suitable computing device.

Each utterance may be processed by apparatus including one or more of anatural language processor, a signal extractor, an utterance sentimentclassifier and a sequential neural network classifier. The processing ofthe utterance may return to the conversation manager a sentiment scoreand a label. The conversation manager may identify a response to theutterance based at least in part on the sentiment score and the label.The conversation manager may transmit the response to the interactiveresponse system. The interactive response system may output the responseto the user. The response may be a voice message, text message, aselectable icon, a graphical user interface display, or any othersuitable message.

The apparatus and methods may provide pre-processing of the utteranceprior to feeding utterance-related data to the sequential neural networkclassifier. The pre-processing may be executed by the natural languageprocessor, the signal extractor and/or the utterance sentimentclassifier. The sequential neural network classifier may executeconversation sentiment scoring and output the sentiment score.

The methods may include the conversation manager receiving the request.In some embodiments, the request may be included in a statelessapplication programming interface (“API”) request. It is to beunderstood that, when the request is described as a stateless APIrequest herein, the invention includes embodiments where the request isan electronic request different from a stateless API request.

In some embodiments, the interactive response system may be an API. TheAPI may be a cloud-based API running on one or more servers. The one ormore servers may run on the internet.

The user device may access, or run, the cloud-based API. For example,when the interactive response system is a chatbot, stateless APIrequests detailed herein may be transferred between the cloud-based APIrunning on the user device and the conversation manager. As such, aresponse to the customer's utterance may be transmitted from theconversation manager to the API, and then output by the API on thecustomer's device.

When the interactive response system is an interactive voice response(“IVR”) system, the user device may connect to the IVR system via atelephone line. The telephony line may input telephony data into the IVRsystem. In these embodiments, stateless API requests detailed herein maybe transferred between the IVR system and the conversation manager. Assuch, a response to the customer's utterance may be transmitted from theconversation manager to the IVR system, and then output by the IVRsystem by generating a voice message to the customer device via thetelephone line.

The interactive response system may receive the utterance from the uservia a telephony network or an internet connection. When the utterance isa first utterance of the user during an interaction between the user andthe interactive response system, the interactive response system maygenerate a stateless API request. The stateless API request may includethe utterance. The stateless API request may be associated with theinteraction by including a tag, identifying caller/user data, an IPaddress, or any other suitable user-identifying data.

When the utterance is not the first utterance during the interaction,the interactive response system may augment a stateless API requestresiding on the interactive response system with the utterance. Thestateless API request being augmented may be a request associated withthe interaction. The stateless API request may include informationrelating to previous utterances expressed by the user during theinteraction, and, in some embodiments, previous responses generated bythe interactive response system and output to the user. Exemplaryinformation stored by the stateless API request may include previousutterance data and a sequence of labels. It is to be understood that‘augmenting’ a request may refer to the adding of data to a request.

The interactive response system may transmit the API request, includingthe utterance, to the conversation manager. The conversation manager mayreceive the API request.

When the request is a first request generated by the interactiveresponse system during the interaction, the conversation manager mayquery one or more databases to retrieve a user profile. The conversationmanager may then augment the API request with the user profile. The userprofile may be retrieved by the conversation manager using identifyingdata input by the user into the interactive response system, such as anaccount number, name, address, and/or any other optional or requiredinformation. The user profile may be retrieved by the conversationmanager using user-identifying data input into the API request by theinteractive response system. The profile of the user may include one ormore of a username, user account information and/or any other suitableuser data.

When the request is a second, third or subsequent API request generatedby the interactive response system during the interaction, the APIrequest may include previous utterance data and the sequence of labels.The sequence of labels may be part of the previous utterance data. Eachlabel in the sequence of labels may be associated with a previousutterance.

The previous utterance data may include, for each previous utteranceexpressed by the user during the interaction, the previous utterance, anintent of the previous utterance, one or more parameters associated withthe previous utterance, one or more semantic meanings associated withthe previous utterance, an input mode of the previous utterance, a labelassociated with the previous utterance, a sentiment score associatedwith the previous utterance, and/or any other suitable previousutterance data.

The input mode may be how the utterance was received by the interactiveresponse system. Exemplary input modes may include voice, text, iconselection, or any other suitable input mode.

The intent of the utterance may be a goal of the user identified in theutterance by the natural language processor. When the interactiveresponse system is a banking application, exemplary goals may be“transfer,” “open new account,” “auto-pay,” “order checks,” etc.

The utterance parameter be one or more words or phrases included in theutterance which are associated with the utterance intent. For example,for an utterance “send 100 dollars to my checking account”, theutterance intent may be ‘transfer’ and the associated parameters may be“100 dollars” and “checking account.”

The semantic meaning of the utterance may be a concept within theutterance and/or one or more relationships between words in theutterance. Semantic meaning may be assigned to an utterance by naturallanguage processor as described herein.

In some embodiments, the previous utterance data may include datarelating to all previous utterances expressed by the user during theinteraction between the user and the interactive response system. Theprevious utterance data may include, for each previous utterance, one ormore pieces of the previous utterance data described herein.

In some embodiments, the previous utterance data may include datarelating to a predetermined number of previous utterances immediatelypreceding the utterance. For example, the previous utterance data mayinclude data relating to up to ten previous utterances expressed by theuser prior to the current utterance (referred to herein as ‘theutterance’). The previous utterance data may include, for each previousutterance, one or more pieces of the previous utterance data describedherein. Thus, when an eleventh utterance is expressed, the interactiveresponse system may purge from the stateless API request the firstutterance and first utterance related data and add the eleventhutterance to the stateless API request.

The methods may include the conversation manager transmitting theutterance to the natural language processor. In some embodiments, theconversation manager may transmit the utterance to the natural languageprocessor by transmitting the stateless API request to the naturallanguage processor. In other embodiments, the utterance may betransmitted to the natural language processor in a separate datatransmission. In some embodiments, previous utterance data may also betransmitted to the natural language processor using either of theaforementioned methods.

The methods may include the natural language processor processing theutterance to output an utterance intent. In some embodiments, themethods may also include the natural language processor processing theutterance to output one or more semantic meanings of the utteranceand/or one or more utterance parameters. The natural language processormay use rules and text matching to identify the utterance intent andutterance parameters. For example, an utterance with the text ‘transfer’may be associated with an intent of ‘transfer.’ Text matching may beused to identify this intent. When the utterance is associated with the‘transfer’ intent, rules may dictate what parameters to extract from theutterance, such as an account name and a dollar amount. Thus, rules andtext matching may be used to identify the utterance parameters. Semanticmeaning may be identified using relationships between the words based onknown natural language processing methods.

The methods may include the natural language processor transmitting theoutput data to one or both of the conversation manager and the signalextractor. In some embodiments, the natural language processor mayupdate the stateless API request to include the output data and transmitthe updated API request to one or both of the conversation manager andthe signal extractor. In other embodiments, the natural languageprocessor may transmit the output data in a data packet to one or bothof the conversation manager and the signal extractor. When the data istransmitted to the conversation manager in a data transmission differentfrom the stateless API request, the conversation manager may add thedata to the stateless API request.

The methods may include the signal extractor receiving the data outputby the natural language processor. The methods may also include thesignal extractor receiving the previous utterance data. When thestateless API request is updated with the natural language processoroutput data and transmitted to the signal extractor, the signalextractor may pull from the stateless API request the NLP output dataand the stored previous utterance data. In other embodiments, the signalextractor may receive the previous utterance data from the conversationmanager. In embodiments where the natural language processor transmitsthe output data back to the conversation manager and not to the signalextractor, the conversation manager may transmit the output utterancedata and the utterance related data directly to the signal extractor viathe stateless API request or a separate data transmission.

The signal extractor may process the utterance data output by thenatural language processor and previous utterance data using logic-basedtext matching, rules and/or artificial intelligence to generate one ormore utterance signals.

A first exemplary workflow of a first signal of the signal extractor mayinclude the signal extractor generating a first utterance signal inresponse to a determination that the utterance intent is identical to anintent of an immediately preceding utterance. The first utterance signalmay be ‘SI=Intent_Repeated(1StepBack).’ A second exemplary workflow forthe signal extractor to generate a second signal may be the signalextractor outputting a second utterance signal such as‘SI=Input-PartialMatch’ when the utterance is determined to be a sub orsuperstring of a previous utterance.

A third exemplary workflow for the signal extractor to generate a thirdsignal may be the signal extractor outputting a third signal in responseto a determination that the semantic meaning of the utterance issubstantially similar to a semantic meaning associated with a previousutterance.

The signal extractor may output a signal each time the signalextractor's pre-stored rules, text matching, and/or AI conditions aremet. Thus, the signal extractor may output one utterance signal or aplurality of utterance signals.

The signal extractor may transmit the utterance signals directly to theutterance sentiment classifier either by an electronic data transmissionincluding the utterance signals or by updating the stateless API requestto include the utterance signals and transmitting the updated statelessAPI request to the utterance sentiment classifier. When the utterancesignals are not added to the stateless API request, the signal extractormay transmit the utterance signals to one or both of the conversationmanager and the utterance sentiment classifier. In embodiments that theconversation manager receives the utterance signals, the conversationmanager may update the stateless API request to include the utterancesignals.

The methods may include the utterance sentiment classifier receiving thesignals. The signals may be received directly from the signal extractorusing methods described above, or from the conversation manager.

The methods may include the utterance sentiment classifier storing ahierarchy of rules in a database. Each stored rule may be associatedwith one or more rule signals and a label. In response to receiving theutterance signals from the signal extractor, the utterance sentimentclassifier may iterate through the hierarchy of rules in sequentialorder to identify a rule in the hierarchy for which the utterancesignals are a superset of the rule's one or more rule signals. Theiterating may end when a rule is identified. As such, although two ormore rules in the hierarchy may satisfy the aforementioned conditions,only the rule highest up in the hierarchy will be identified by theiterating. Any suitable number of rules may be stored by the hierarchy.

The label associated with the identified rule may be transmitted to oneor both of the sequential neural network classifier and the conversationmanager. The label may be added to the stateless API request and therequest transmitted to the sequential neural network classifier, or thelabel may be transmitted in an electronic transmission not including thestateless API request. The sequence of labels may be received by thesequential neural network classifier from either the utterance sentimentclassifier or from the conversation manager. For example, when utterancesentiment classifier transmits the label to the conversation manager,the conversation manager may transmit the label and the sequence of thelabels to the classifier.

The methods may also include the sequential neural network classifierreceiving a data input including the sequence of labels and a labelassociated with the rule identified by the utterance sentimentclassifier using methods described above. The label may be received fromthe utterance sentiment classifier or the conversation manager. Thesequence of labels may be received from the conversation manager orextracted from the stateless API request. In some embodiments, the datainput to the sequential neural network classifier may not include theutterance. The sequential neural network classifier may be any suitabledeep learning machine learning algorithm, such as an LS™ algorithm.

The methods may include the sequential neural network classifierprocessing the data input using a trained algorithm. The methods mayfurther include the sequential neural network outputting the sentimentscore. The sentiment score may be a score included in a range of scores,such as −2 to +2, 0 to 5, 0 to 100, −5 to +5 or any other suitable scorerange.

In some embodiments, the sequential neural network classifier maytransmit the sentiment score to the conversation manager. In someembodiments, sequential neural network classifier may add the sentimentscore to the stateless API request and subsequently transmit thestateless API request to the conversation manager.

The methods may include the conversation manager receiving the label andthe sentiment score. The label and the sentiment score may be receivedby the conversation manager in any of the methods described herein. Themethods may include the conversation manager identifying a response tothe utterance based on the utterance intent, the label and the sentimentscore.

The response may be identified using a plurality of rules, each rulebeing associated with one or more of the intent, label and/or sentimentscore. For example, a response of ‘would you like to transfer to accountX or account Y’ may be identified based on a first rule stating thatwhen the intent is ‘transfer’ and the semantic score is above athreshold value. The utterance intent may be received by theconversation manager in any of the methods described herein. However, ifthe intent is ‘transfer’ but the semantic score is below a thresholdvalue, indicating a negative user experience during the interaction, asecond rule may be used identifying a response of ‘would you like me toconnect you to customer service.’

In some embodiments, the methods may include the conversation managertransmitting the response to the interactive response system. In some ofthese embodiments, the methods may include the interactive responsesystem receiving the response and outputting the response to the user.

In other embodiments, after the conversation manager identifies theresponse, the conversation manager may augment the stateless API requestto include the response. In embodiments where the API request has notbeen previously augmented with one or more of the utterance intent, thesemantic meaning, the utterance parameter, the label and the sentimentscore, the conversation manager may augment the stateless API requestwith the response and any of the aforementioned data not yet input fromthe stateless API request.

In some of these embodiments, after the augmenting, the methods mayinclude transmitting the stateless API request to the user device.Transmission of the stateless API request to the user device maycomprise transmitting the stateless API request to the interactiveresponse system in communication with the user device. The interactiveresponse system may receive the stateless API request and output theresponse to the user.

When the interactive response system is an IVR system, the transmittingthe response to the user may include the IVR system generating an audiomessage to the user. In some of these embodiments, the utterance may bea verbal message expressed to the user by the IVR system.

When the interactive response system is a chatbot, the transmitting theresponse to the user may include the chatbot generating a text message,generating an audio message, or displaying a selectable icon on agraphical user interface of the user device.

In some embodiments, the methods may further comprise training thesequential neural network prior to the sequential neural networkreceiving the data input. The training may include feeding an untrainedalgorithm with multiple sequences of labels. Each label stored in eachsequence of labels may be associated with a sentiment score. Thetraining may transform the untrained algorithm to the trained algorithm.

The pre-processing of the utterance by the natural language processor,the signal extractor and the utterance sentiment classifier may reducethe sequential neural network classifier's data input to the label andthe sequence of labels. Thus, the classifier assigns the sentiment scoreto the conversation based on label data only and not data comprising theutterances. This may increase a speed at which the classifier returnsthe sentiment score relative to a speed needed by the classifier toprocess data strings including the utterance and the previousutterances. The feeding of the classifier with label data may alsodecrease resources consumed by the sequential neural network classifierwhen processing the data input compared to larger resources needed toprocess utterances themselves.

Furthermore, training the classifier is simplified by requiring onlysets of labels and their associated scores, instead of large volumes ofall utterances that may be expressed by a user during a conversation anda value associated with each of the utterances.

The aforementioned increase in processing speed may allow the sequentialneural network classifier to output a sentiment score fast enough sothat the conversation manager can use the sentiment score when selectinga response to the user in real-time.

The systems and methods of the invention may include apparatus forproviding the pre-processing of the utterance as described herein. Theapparatus may include the conversation manager, the natural languageprocessor, the signal extractor, the utterance sentiment classifier andthe sequential neural network classifier. Each of the aforementionedapparatus may perform functions, and have characteristics, as describedherein. It is to be understood that apparatus “for” performing afunction is apparatus “configured to” perform the function.

The apparatus may include the conversation manager. The conversationmanage may include a first processor for receiving the request. Therequest may be a stateless API request. The request may include theutterance and previous utterance data. The previous utterance data mayinclude the sequence of labels and any other previous utterance datadescribed herein. Each label in the sequence of labels may be associatedwith a previous utterance expressed by a user during the interaction.

The apparatus may also include the natural language processor forprocessing the utterance to output the utterance intent, the semanticmeaning of the utterance and the utterance parameter. The utteranceparameter may include one or more words included in the utterance andassociated with the utterance intent.

The apparatus may also include the signal extractor for processing theutterance, the utterance intent, the semantic meaning, the utteranceparameter, and the previous utterance data to generate one or moreutterance signals. The signal extractor may include one or moreprocessors.

The apparatus may further include the utterance sentiment classifier.The utterance sentiment classifier may include a memory for storing ahierarchy of rules, each rule being associated with one or more rulesignals and a label. The utterance sentiment classifier may include asecond processor for, in response to receiving the one or more utterancesignals from the signal extractor, iterating through the hierarchy ofrules in sequential order to identify a rule in the hierarchy for whichthe one or more utterance signals are a superset of the rule's one ormore rule signals, the iterating ending when the rule is identified.

The apparatus may additionally include the sequential neural networkclassifier for receiving the data input including the sequence of labelsand the label associated with the rule identified by the utterancesentiment classifier. The data input may not include the utterance. thesequential neural network classifier may include one or more processors.

The sequential neural network classifier may be configured to processthe data input using a trained algorithm. The sequential neural networkclassifier may be further configured to output a sentiment score.

The conversation manager may be configured to identify a response to theutterance based on the utterance intent, the label and the sentimentscore. The conversation manager may be further configured to augment thestateless API request to include the response, the utterance intent, thesemantic meaning, the utterance parameter, the label and the sentimentscore. After the augmenting, the conversation manage may be additionallyconfigured to transmit the stateless API request to the interactiveresponse system/

The apparatus may include the interactive response system. Theinteractive response system may be configured to receive the statelessAPI request from the conversation manager and output the response to theuser.

The apparatus may include the third processor for feeding the sequentialneural network with training data prior to the sequential neural networkreceiving the data input. This third processor may be configured to theexecute the training, the executing including feeding an untrainedalgorithm with multiple sequences of labels. Each label stored in eachsequence of labels may be associated with a sentiment score. Thetraining may transform the untrained algorithm to the trained algorithm.

Methods for providing pre-processing of an utterance prior to feedingutterance-related data to a sequential neural network classifier forconversation sentiment scoring, the utterance being expressed, by auser, to an interactive response system during an interaction betweenthe user and the interactive response system are provided. The methodsmay include using a conversation manager to receive a statelessapplication programming interface (“API”) request. The requestpreferably includes the utterance, previous utterance data and asequence of labels. Each label may be associated with a previousutterance expressed by a user during the interaction.

The methods may further include using a natural language processor toprocess the utterance to output an utterance intent, a semantic meaningof the utterance and an utterance parameter. The utterance parameter mayinclude one or more words included in the utterance. The utteranceparameter may further be associated with the intent.

The method may also include using a signal extractor to process theutterance, the utterance intent, the semantic meaning, the utteranceparameter, and the previous utterance data to obtain one or moreutterance signals. The signal extractor may also be used for checkingthe one or more utterance signals to determine whether, during theinteraction, the one or more utterance signals matches apreviously-determined, preferably most probable, utterance signal—wherethe utterance signal reflects a most probable user intent.

To the extent that the one or more utterance signals matches apreviously-determined utterance signal, then the signal extractor maydetermine whether the one or more utterance signals obtained, previouslyduring the interaction, failed to satisfy the user. When the one or moreutterance signals matches a previous utterance signal and the sentimentscore reflects user di satisfaction, then the signal extractor may shiftthe predicted intent, which will change the utterance signal, to thenext most probable intent. The next most probably intent may then beused for generating one or more alternate utterance signals. Toreiterate, generating one or more alternate utterance signals may, incertain embodiments, involve shifting from a highest-probabilityutterance signal to a next-highest-probability utterance signal. Thisexemplary shift from a highest-probability utterance signal to anext-highest-probability utterance signal is set forth in more detailbelow.

The following is one example of a process in which the utterance signalis shifted from a first, most probable intent, to a second, next mostprobable intent. Given an utterance, the NLP engine may predict aprobability distribution over a number of intents. The probabilitydistribution might be the following: [GET_ACCOUNT_NUMBER=0.65,GET_ROUTING_NUMBER=0.30, OPEN_AN_ACCOUNT=0.04, . . . ]. The NLP enginemay inform the conversation manager that the correct intent is the mostprobable one (in this case, GET_ACCOUNT_NUMBER). Embodiments set forthherein may indicate that the sentiment component (referred to herein, inthe alternative, as the sentiment extractor) may detect that thesentiment component has recently—e.g., in the same customerinteraction—already predicted GET_ACCOUNT_NUMBER (without success).Instead of repeating its previous error, the sentiment component maychange the predicted intent to the GET_ROUTING_NUMBER. Predicting theGET_ROUTING_NUMBER intent preferably changes the extracted signal(s) atleast for the reason that the sentiment component would no longer havedetected a signal that the same intent got negatively predictedrecently.

Further, the method may include using an utterance sentiment classifierto store a hierarchy of rules. Each rule in the hierarchy of rules maybe associated with one or more rule signals and a label. In response toreceiving the one or more utterance signals, or alternate utterancesignals, from the signal extractor, the method may include iteratingthrough the hierarchy of rules in sequential order to identify a firstrule in the hierarchy for which the one or more utterance signals are asuperset of the first rule's one or more rule signals.

The sequential neural network classifier may be used for receiving adata input including the sequence of labels and a label associated withthe rule identified by the utterance sentiment classifier. The datainput may not include the utterance. The sequential neural networkclassifier may also be used for processing the data input using atrained algorithm and for outputting a sentiment score.

A conversation manager may be used for identifying a response to theuser utterance based on the utterance intent, the label and thesentiment score; augmenting the stateless API request to include theresponse, the utterance intent, the semantic meaning, the utteranceparameter, the label and the sentiment score; and, after the augmenting,transmitting the stateless API request to the interactive responsesystem.

In some embodiments, the interactive response system may finally be usedto receive the stateless API request and output the response to theuser.

The pre-processing of the utterance by the natural language processor,the signal extractor and the utterance sentiment classifier maypreferably reduce the data input to the label and the sequence oflabels, thereby increasing a speed at which the sequential neuralnetwork classifier returns the sentiment score and decreasing resourcesconsumed by the sequential neural network classifier when processing thedata input.

In some embodiments, in response to a determination that a storedprevious utterance is associated with the first rule and the sentimentscore associated with the previous utterance is above the thresholdvalue, the method may transmit the label associated with the first ruleto the sequential neural network classifier.

In some embodiments, in response to a determination that no storedprevious utterance is associated with any rules, the method may includeterminating the iterating and transmitting the label associated with thefirst rule to the sequential neural network classifier.

Apparatus and methods described herein are illustrative. Apparatus andmethods in accordance with this disclosure will now be described inconnection with the figures, which form a part hereof. The figures showillustrative features of apparatus and method steps in accordance withthe principles of this disclosure. It is to be understood that otherembodiments may be utilized and that structural, functional andprocedural modifications may be made without departing from the scopeand spirit of the present disclosure.

The steps of methods may be performed in an order other than the ordershown or described herein. Embodiments may omit steps shown or describedin connection with illustrative methods. Embodiments may include stepsthat are neither shown nor described in connection with illustrativemethods.

Illustrative method steps may be combined. For example, an illustrativemethod may include steps shown in connection with another illustrativemethod.

Apparatus may omit features shown or described in connection withillustrative apparatus. Embodiments may include features that areneither shown nor described in connection with the illustrativeapparatus. Features of illustrative apparatus may be combined. Forexample, an illustrative embodiment may include features shown inconnection with another illustrative embodiment.

Illustrative embodiments of apparatus and methods in accordance with theprinciples of the invention will now be described with reference to theaccompanying drawings, which form a part hereof. It is to be understoodthat other embodiments may be utilized, and structural, functional andprocedural modifications may be made without departing from the scopeand spirit of the present invention.

The drawings show illustrative features of apparatus and methods inaccordance with the principles of the invention. The features areillustrated in the context of selected embodiments. It will beunderstood that features shown in connection with one of the embodimentsmay be practiced in accordance with the principles of the inventionalong with features shown in connection with another of the embodiments.

One of ordinary skill in the art will appreciate that the steps shownand described herein may be performed in other than the recited orderand that one or more steps illustrated may be optional. The methods ofthe above-referenced embodiments may involve the use of any suitableelements, steps, computer-executable instructions, or computer-readabledata structures. In this regard, other embodiments are disclosed hereinas well that can be partially or wholly implemented on acomputer-readable medium, for example, by storing computer-executableinstructions or programs or by utilizing computer-readable datastructures.

FIG. 1 shows illustrative apparatus and methods in accordance with theinvention. Apparatus illustrated in FIG. 1 includes user device 101,interactive response system 102, conversation manager 103, naturallanguage processor (“NLP”) 105, signal extractor 107, utterancesentiment classifier 109, sequential neural network classifier 111 anddatabase storing training data 113.

The apparatus illustrated in FIG. 1 includes arrows representing a flowof data between the apparatus. In FIG. 1 , conversation manager 103 isillustrated as transmitting data to NLP 105 and receiving data fromsequential neural network classifier 111. Conversation manager may alsosupport communications allowing for data transmission and/or receipt tosome or all of NLP 105, signal extractor 107, utterance sentimentclassifier 109, sequential neural network classifier 111 and databasestoring training data 113. Additional data connections may beestablished between some or all of NLP 105, signal extractor 107,utterance sentiment classifier 109, sequential neural network classifier111 and database storing training data 113.

Conversation manager 103 may also be in communication with one or moreuser databases (not shown). Conversation manager may retrieve from thesedatabase(s) user data, such as, for example, user identifyinginformation, user account information, and any other suitable data.

User device 101 is illustrated transmitting utterance 104 to interactiveresponse system 102. Interactive response system 102 may be acloud-based API running on the user device, an API accessed by the userdevice through an internet connection, an interactive voice responsesystem communicating with the user device via a telephony line, or anyother suitable system. Interactive response system 102 may add thereceived utterance to stateless API request 108 and transmit statelessAPI request 108 to conversation manager 103. Utterance 104 may beexpressed by a user during an interaction with an interactive responsesystem as described herein.

Conversation manager 103 may transmit utterance 104 to NLP 105. NLP 105may process utterance 104 to output an utterance intent, and otherrelated data associated with utterance 104, such as semantic meaning andone or more utterance parameters. The output utterance data and, in someembodiments, utterance 104, may be transmitted to signal extractor 107by NLP 105 or conversation manager 103.

Signal extractor 107 may receive utterance 104 and utterance data fromNLP 105. Signal extractor 107 may also receive previous utterance data117. Previous utterance data 117 may be fed to signal extractor 107 byNLP 105 or conversation manager 103.

Signal extractor 107 may process the data output by NLP 105 and previousutterance data 117. Signal extractor 107 may use one or more rules,algorithms, and/or ML to identify one or more signals.

The one or more signals may be fed to utterance sentiment classifier109. Utterance sentiment classifier may store a plurality of rules,ordered in a hierarchy, each rule being associated with one or moresignals S1 . . . SN and a label. Utterance sentiment classifier 109 mayuse the received signals S1 . . . SN to identify a rule in the storedrules-based hierarchy for which the received signals are a superset ofthe identified rule's signals.

Utterance sentiment classifier 109 may output label USi of theidentified rule to sequential neural network classifier 111. Sequentialneural network classifier 111 may receive label USi from utterancesentiment classifier 109. Sequential neural network classifier 111 mayalso receive sequence 119 of labels US1 . . . USi−1 from utterancesentiment classifier 109 or conversation manager 103. Sequence 119 oflabels US1 . . . USi−1 may be an ordered set of historical labelsassociated with previous utterances uttered by the user during theinteraction with the interactive response system. Sequence 119 of labelsUS1 . . . USi−1 may include the historical labels ordered, in sequence,from the first or earliest generated label in the interaction to thelast, or latest, generated label in the interaction. The latestgenerated label may be a label generated immediately preceding labelUSi.

Sequential neural network classifier 111 may use a trained algorithmidentify a sentiment score based on input labels US1 . . . USi. Thetrained algorithm may output a sentiment score CSi based on theprocessing of input labels US1 . . . USi−1. The trained algorithm may becreated by feeding an untrained algorithm with a plurality of manuallylabeled conversations. The manually labeled conversations may be storedin database storing training data 113. The manually labeledconversations may include a sequence of labels extracted from aconversation, with each label in the sequence being assigned a sentimentscore.

In some embodiments, utterance sentiment classifier 109 and sequentialneural network classifier 111 may be part of the same component 115. Inother embodiments, utterance sentiment classifier 109 and sequentialneural network classifier 111 may be separate components.

Sequential neural network classifier 111 may output sentiment score CSito conversation manager 103. In some embodiments, sequential neuralnetwork classifier 111 may also transmit to conversation manager 103label USi. In other embodiments, utterance sentiment classifier 109 maytransmit label USi to conversation manager 103.

Conversation manager 103 may receive sentiment score CSi and label USi.Conversation manager 103 may feed this data, in addition to theutterance intent generated by NLP 105, to one or more rules-basedalgorithms and/or ML algorithms to identify response 106 to utterance104. Conversation manager 103 may then transmit response 106 tointeractive response system 102 via stateless API request 108.Interactive response system may receive stateless API request 108 fromconversation manager 103, and subsequently transmit response 106 to userdevice 101. The response may be included in the stateless API requestreceived by conversation manager 103. The stateless API request may beaugmented not only with the response, but also with some or all ofsentiment score CSi, label USi, utterance intent, semantic meaning,utterance parameters and/or signals S1 . . . SN.

FIG. 2 shows illustrative diagram 201 in accordance with the invention.In diagram 201, illustrative rules 202 are shown. One or more ofillustrative rules 202 may be included in the utterance sentimentclassifier's hierarchy of rules.

Each of rules 202 are associated with a label 203 and a signal 205. Indiagram 201, Rule R1 is associated with the label ‘upset’ and the signal‘upset.’ Signal ‘upset’ may be output by the signal extractor when thetext of the utterance includes a term such as ‘upset’ or another termshowing upset emotions. Rule R2 is associated with the label ‘affirm’and the signal ‘YES.’ Signal ‘YES’ may be output by the signal extractorwhen the utterance includes the term ‘yes’ or something similar. Rule R3is associated with the label ‘deny’ and the signal ‘NO.’ Signal ‘NO’ maybe output by the signal extractor when the utterance includes the word‘NO’ or something similar. Rule R4 is associated with the label‘DidNotUnderstand(HelpSuggestion)’ and the signal‘CurrentIntent-SI_Help_Suggestions,Signal=Input-Repeated(DifferentIntent).’ The aforementioned signal maybe output by the signal extractor when the utterance is nearly identicalto a previous utterance, and the previous utterance has a differentintent then the previous utterance.

FIG. 3 shows illustrative diagram 301. Illustrative diagram 301 includesan exemplary list of sentiment scores and their associated definitions.Diagram 301 includes first column 303 having header ‘Sentiment Score’and second column 305 having header ‘Definition of Sentiment Score.’ Thesentiment scores illustrated in diagram 301 show an illustrative rangeof sentiment scores that may be used by the sequential neural networkclassifier to classify an utterance.

In diagram 301, row 307 includes a sentiment score of ‘−2’ and anassociated definition of ‘Very negative.’ Row 309 includes a sentimentscore of ‘−1’ and an associated definition of ‘Negative, frustrationfrom the number of steps, repeated process.’ Row 311 includes asentiment score of ‘0’ and an associated definition of ‘Mid-flows andcases not sure.’ Row 313 includes a sentiment score of ‘1’ and anassociated definition of ‘Conversation goes smooth.’ Row 315 includes asentiment score of ‘2’ and an associated definition of ‘Very Positive.’

FIGS. 4A-4B show an exemplary interaction between a user and aninteractive response system in diagram 401. The exemplary interactionillustrated in diagram 401 is an interaction for which the sequentialneural network classifier assigned positive sentiment scores. Datadisplayed in diagram 401 for each interaction includes utterance 403,input mode 405, response 407, label 409 and sentiment score 411.

In diagram 401, input mode 405 is either text or gesture. An input modeof ‘text’ may refer to a user inputting text via typing on a computer orsmartphone during a chat session with the interactive response system.An input mode of ‘gesture’ may refer to a user selecting an icondisplayed on his computing device. For example, response ‘no problem, towhich account?’ may also include displaying to the user multipleselectable icons, each icon identifying one of the user's accounts. Theuser's next utterance ‘Advantage Savings****’ associated with the inputmode ‘gesture’ may be a selection, by the user, of a displayedselectable icon entitled ‘Advantage Savings****.’

Labels displayed in the label 409 column may be exemplary labelsassigned to the displayed utterances by the utterance sentimentclassifier. Scores displayed in the sentiment score 411 column may bescores assigned to the displayed utterances by the sequential neuralnetwork classifier.

FIGS. 5A-5B show an exemplary interaction between a user and aninteractive response system in diagram 501. The exemplary interactionillustrated in diagram 501 is an interaction for which the sequentialneural network classifier assigned negative sentiment scores. Datadisplayed in diagram 501 for each interaction includes utterance 503,response 505, label 507 and sentiment score 509.

An input mode is not displayed in diagram 501. The exemplary interactionillustrated in diagram 501 may be an interaction between a user and aninteractive voice response system. As such, each of the user'sutterances may be vocal utterances.

Labels displayed in the label 507 column may be exemplary labelsassigned to the displayed utterances by the utterance sentimentclassifier. Scores displayed in the sentiment score 509 column may bescores assigned to the displayed utterances by the sequential neuralnetwork classifier.

FIG. 6 shows a portion of an exemplary interaction between a user and aninteractive response system in diagram 601. The exemplary interactionillustrated in diagram 601 may be an interaction between a user and aninteractive voice response system. For each interaction between the userand the interactive response system, diagram 601 illustrates thefollowing data: utterance 603, response 605, label 607 and sentimentscore 609.

In diagram 601, deflection point 611 is identified. In FIG. 6 ,deflection point 611 is an identified exchange between the user and thesystem, during the interaction, before which the sentiment scores werepositive/zero valued, and after which the sentiment score were negativevalues. Identification of deflection point 611 may assist theconversation manager in identifying suboptimal responses by theinteractive response system and draw attention to workflows that mayneed to be fixed, deleted or updated.

Labels displayed in the label 607 column may be exemplary labelsassigned to the displayed utterances by the utterance sentimentclassifier. Scores displayed in the sentiment score 609 column may bescores assigned to the displayed utterances by the sequential neuralnetwork classifier.

FIG. 7 shows a portion of an exemplary interaction between a user and aninteractive response system in diagram 701. The exemplary interactionillustrated in diagram 701 may be an interaction between a user and aninteractive voice response system. For each interaction between the userand the interactive response system, diagram 701 illustrates thefollowing data: utterance 703, label 705, sentiment score 707 andsignals 709.

In diagram 701, deflection point 711 is identified. In FIG. 7 ,deflection point 711 is a signal generated by the signal extractor foran utterance immediately prior to an utterance request, by the user, forcustomer service. The signal at deflection point 711 is‘Signal=Input-Similar(1StepsBack).’ This signal may indicate thatutterance processed by the signal extractor (i.e., “What location did Iopen my account) is similar to an utterance immediately preceding theprocessed utterance (i.e., “What location did I open my account”).Repetition, by a user, of an idea immediately preceding a request tocontact customer service may be information useful to the customerservice agent when the call is transferred. This information may bedisplayed to the customer service representative when the call istransferred or, if the call is not transferred, stored in a database forfuture analysis. Utterances that are repeated prior to a request forcustomer service may identify a suboptimal response by the interactiveresponse system and draw attention to workflows that may need to befixed, deleted or updated.

Labels displayed in the label 705 column may be exemplary labelsassigned to the displayed utterances by the utterance sentimentclassifier. Scores displayed in the sentiment score 707 column may bescores assigned to the displayed utterances by the sequential neuralnetwork classifier. Signals displayed in signals 709 may be exemplarysignals applied to an utterance by the signal extractor.

FIG. 8 shows an illustrative block diagram of system 800 that includescomputer 801. Computer 801 may alternatively be referred to herein as an“engine,” “server” or a “computing device.” Computer 801 may be aworkstation, desktop, laptop, tablet, smart phone, or any other suitablecomputing device. Elements of system 800, including computer 801, may beused to implement various aspects of the systems and methods disclosedherein. Each of the apparatus illustrated in FIG. 1 , including userdevice 101, interactive response system 102, conversation manager 103,NLP 105, signal extractor 107, utterance sentiment classifier 109 andsequential neural network classifier 111, may include some or all of theelements and apparatus of system 800.

Computer 801 may have a processor 803 for controlling the operation ofthe device and its associated components, and may include RAM 805, ROM807, input/output circuit 809, and a non-transitory or non-volatilememory 815. Machine-readable memory may be configured to storeinformation in machine-readable data structures. The processor 803 mayalso execute all software running on the computer—e.g., the operatingsystem and/or voice recognition software. Other components commonly usedfor computers, such as EEPROM or Flash memory or any other suitablecomponents, may also be part of the computer 801.

The memory 815 may be comprised of any suitable permanent storagetechnology—e.g., a hard drive. The memory 815 may store softwareincluding the operating system 817 and application(s) 819 along with anydata 811 needed for the operation of the system 800. Memory 815 may alsostore videos, text, and/or audio assistance files. The data stored inMemory 815 may also be stored in cache memory, or any other suitablememory.

Input/output (“I/O”) module 809 may include connectivity to amicrophone, keyboard, touch screen, mouse, and/or stylus through whichinput may be provided into computer 801. The input may include inputrelating to cursor movement. The input/output module may also includeone or more speakers for providing audio output and a video displaydevice for providing textual, audio, audiovisual, and/or graphicaloutput. The input and output may be related to computer applicationfunctionality.

System 800 may be connected to other systems via a local area network(LAN) interface 813. System 800 may operate in a networked environmentsupporting connections to one or more remote computers, such asterminals 841 and 851. Terminals 841 and 851 may be personal computersor servers that include many or all of the elements described aboverelative to system 800. The network connections depicted in FIG. 8include a local area network (LAN) 825 and a wide area network (WAN)829, but may also include other networks. When used in a LAN networkingenvironment, computer 801 is connected to LAN 825 through a LANinterface 813 or an adapter. When used in a WAN networking environment,computer 801 may include a modem 827 or other means for establishingcommunications over WAN 829, such as Internet 831. Connections betweenSystem 800 and Terminals 851 and/or 841 may be used for conversationmanager 103 to communicate with user device 101.

It will be appreciated that the network connections shown areillustrative and other means of establishing a communications linkbetween computers may be used. The existence of various well-knownprotocols such as TCP/IP, Ethernet, FTP, HTTP and the like is presumed,and the system can be operated in a client-server configuration topermit retrieval of data from a web-based server or API. Web-based, forthe purposes of this application, is to be understood to include acloud-based system. The web-based server may transmit data to any othersuitable computer system. The web-based server may also sendcomputer-readable instructions, together with the data, to any suitablecomputer system. The computer-readable instructions may be to store thedata in cache memory, the hard drive, secondary memory, or any othersuitable memory.

Additionally, application program(s) 819, which may be used by computer801, may include computer executable instructions for invokingfunctionality related to communication, such as e-mail, Short MessageService (SMS), and voice input and speech recognition applications.Application program(s) 819 (which may be alternatively referred toherein as “plugins,” “applications,” or “apps”) may include computerexecutable instructions for invoking functionality related to performingvarious tasks. Application programs 819 may utilize one or morealgorithms that process received executable instructions, perform powermanagement routines or other suitable tasks. Application programs 819may utilize one or more decisioning processes for the processing ofcalls received from calling sources as detailed herein.

Application program(s) 819 may include computer executable instructions(alternatively referred to as “programs”). The computer executableinstructions may be embodied in hardware or firmware (not shown). Thecomputer 801 may execute the instructions embodied by the applicationprogram(s) 819 to perform various functions.

Application program(s) 819 may utilize the computer-executableinstructions executed by a processor. Generally, programs includeroutines, programs, objects, components, data structures, etc. thatperform particular tasks or implement particular abstract data types. Acomputing system may be operational with distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, a program may be located in both local and remote computerstorage media including memory storage devices. Computing systems mayrely on a network of remote servers hosted on the Internet to store,manage, and process data (e.g., “cloud computing” and/or “fogcomputing”).

Any information described above in connection with data 811, and anyother suitable information, may be stored in memory 815. One or more ofapplications 819 may include one or more algorithms that may be used toimplement features of the disclosure comprising the processing of theutterance by NLP 105, the extracting of signals by signal extractor 107,the outputting of a label by utterance sentiment classifier 109 and theprocessing of the labels to identify a sentiment score by sequentialneural network classifier 111.

The invention may be described in the context of computer-executableinstructions, such as applications 819, being executed by a computer.Generally, programs include routines, programs, objects, components,data structures, etc., that perform particular tasks or implementparticular data types. The invention may also be practiced indistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, programs may be located in bothlocal and remote computer storage media including memory storagedevices. It should be noted that such programs may be considered, forthe purposes of this application, as engines with respect to theperformance of the particular tasks to which the programs are assigned.

Computer 801 and/or terminals 841 and 851 may also include various othercomponents, such as a battery, speaker, and/or antennas (not shown).Components of computer system 801 may be linked by a system bus,wirelessly or by other suitable interconnections. Components of computersystem 801 may be present on one or more circuit boards. In someembodiments, the components may be integrated into a single chip. Thechip may be silicon-based.

Terminal 851 and/or terminal 841 may be portable devices such as alaptop, cell phone, Blackberry™, tablet, smartphone, or any othercomputing system for receiving, storing, transmitting and/or displayingrelevant information. Terminal 851 and/or terminal 841 may be one ormore user devices. Terminals 851 and 841 may be identical to system 800or different. The differences may be related to hardware componentsand/or software components.

The invention may be operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, tablets, mobile phones, smart phones and/or otherpersonal digital assistants (“PDAs”), multiprocessor systems,microprocessor-based systems, cloud-based systems, programmable consumerelectronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

FIG. 9 shows illustrative apparatus 900 that may be configured inaccordance with the principles of the disclosure. Apparatus 900 may be acomputing device. Apparatus 900 may include one or more features of theapparatus shown in FIG. 8 . Apparatus 900 may include chip module 902,which may include one or more integrated circuits, and which may includelogic configured to perform any other suitable logical operations.

Apparatus 900 may include one or more of the following components: I/Ocircuitry 904, which may include a transmitter device and a receiverdevice and may interface with fiber optic cable, coaxial cable,telephone lines, wireless devices, PHY layer hardware, a keypad/displaycontrol device or any other suitable media or devices; peripheraldevices 906, which may include counter timers, real-time timers,power-on reset generators or any other suitable peripheral devices;logical processing device 908, which may compute data structuralinformation and structural parameters of the data; and machine-readablememory 910.

Machine-readable memory 910 may be configured to store inmachine-readable data structures: machine executable instructions,(which may be alternatively referred to herein as “computerinstructions” or “computer code”), applications such as applications819, signals, and/or any other suitable information or data structures.

Components 902, 904, 906, 908 and 910 may be coupled together by asystem bus or other interconnections 912 and may be present on one ormore circuit boards such as circuit board 920. In some embodiments, thecomponents may be integrated into a single chip. The chip may besilicon-based.

FIG. 10 shows a snippet 1002 from FIG. 1 . FIG. 10 shows element 109 ofFIG. 1 . Element 109, and its operation, is described above in detail.As mentioned above—the sequence of labels US1 . . . USi−1 may be anordered set of historical labels associated with previous utterancesuttered by the user during the interaction with the interactive responsesystem. The sequence of labels US1 . . . USi−1 may include thehistorical labels ordered, in sequence, from the first or earliestgenerated label in the interaction to the last, or latest, generatedlabel in the interaction. The latest generated label may be a labelgenerated immediately preceding label USi.

FIG. 10 also shows element 1004. Element 1004 shows the sequence oflabels a list of additional US1 . . . USi. However, FIG. 10 shows thislist in additional granularity. Specifically, element 1006 shows thislist in additional granularity—i.e., with additional list membersUS2-US4. In the embodiments relating to FIG. 10 (and FIG. 11 ), inresponse to receiving the one or more utterance signals, the classifieriterates through the hierarchy of rules in sequential order to identifya first rule for which the one or more utterance signals are a supersetof the rule's one or more rule signals. This rule typically reflects themost probable intent of the user.

FIGS. 11A-11B shows another illustrative flow diagram in accordance withthe invention. In FIG. 11A, at step 1102, the method shows processingthe utterance, the utterance intent, the semantic meaning, the utteranceparameter, and the previous utterance data to generate one or moreutterance signals. At 1104, FIG. 11A shows that, to the extent that theone or more of the utterance signals, reflective of the most probableuser intent, were already selected in the same user interaction andfailed to satisfy the user (as shown in 1106) then the process generatesone or more alternate signals. The one or more alternate signalspreferably corresponds to the next most probable intent. Following step1106 (at step 1108B) the process shows storing a hierarchy ofrules—wherein each rule in the hierarchy is associated with one or morerules signals and a label and, at 1112, in response to receiving the oneor more alternate utterance signals—iterating through the hierarchy ofrules to identify the first rule in the hierarchy for which the one ormore alternate utterance signals are a superset of the first rule's oneor more rule signals.

If it is determined that when the one or more utterances signals,reflective of the most probable user intent, previously satisfied theuser, then at 1108A, the process shows storing a hierarchy of rules,each rule in the hierarchy of rules being associated with the one ormore rule signals and a label. At 1110, the process shows further, inresponse to receiving the one or more utterance signals, iteratingthrough the hierarchy of rules in sequential order to identify a firstrule in the hierarchy for which the one or more utterance signals are asuperset of the first rule's one or more rule signals.

Thus, methods and apparatus for leveraging sentiment values in flagging,modifying and/or removal of real time workflows are provided. Personsskilled in the art will appreciate that the present invention can bepracticed by other than the described embodiments, which are presentedfor purposes of illustration rather than of limitation. The presentinvention is limited only by the claims that follow.

What is claimed is:
 1. A method for providing pre-processing of anutterance prior to feeding utterance-related data to a sequential neuralnetwork classifier for conversation sentiment scoring, the utterancebeing expressed, by a user, to an interactive response system during aninteraction between the user and the interactive response system, themethod comprising: a conversation manager receiving a statelessapplication programming interface (“API”) request including theutterance, previous utterance data and a sequence of labels, each labelbeing associated with a previous utterance expressed by a user duringthe interaction; a natural language processor processing the utteranceto output an utterance intent, a semantic meaning of the utterance andan utterance parameter, the utterance parameter comprising one or morewords included in the utterance and being associated with the intent; asignal extractor: processing the utterance, the utterance intent, thesemantic meaning, the utterance parameter, and the previous utterancedata to generate one or more utterance signals, wherein the one or moreutterance signals corresponds to a most probable intent of the user;when the one or more utterance signals has been previously generatedduring the interaction and has previously not satisfied the user, thenmodifying the one or more utterance signals to correspond to a next mostprobable intent of the user; and an utterance sentiment classifier:storing a hierarchy of rules, each rule in the hierarchy of rules beingassociated with one or more rule signals and a label; in response toreceiving the one or more utterance signals from the signal extractor,iterating through the hierarchy of rules in sequential order to identifya first rule in the hierarchy for which the one or more utterancesignals are a superset of the first rule's one or more rule signals; andthe sequential neural network classifier: receiving a data inputincluding the sequence of labels and a label associated with the ruleidentified by the utterance sentiment classifier, the data input notincluding the utterance; processing the data input using a trainedalgorithm; and outputting a sentiment score; the conversation manager:identifying a response to the user utterance based on the utteranceintent, the label and the sentiment score; augmenting the stateless APIrequest to include the response, the utterance intent, the semanticmeaning, the utterance parameter, the label and the sentiment score; andafter the augmenting, transmitting the stateless API request to theinteractive response system; and the interactive response systemreceiving the stateless API request and outputting the response to theuser; wherein: the pre-processing of the utterance by the naturallanguage processor, the signal extractor and the utterance sentimentclassifier reduces the data input to the label and the sequence oflabels, thereby increasing a speed at which the sequential neuralnetwork classifier returns the sentiment score and decreasing resourcesconsumed by the sequential neural network classifier when processing thedata input.
 2. The method of claim 1 wherein the previous utterance dataincludes, for each previous utterance expressed by the user during theinteraction: the previous utterance; an intent of the previousutterance; and a parameter of the previous utterance.
 3. The method ofclaim 1 wherein the previous utterance data includes data relating to apredetermined number of previous utterances immediately preceding theutterance, the previous utterance data, for each previous utterance,including: the previous utterance; an intent of the previous utterances;and a term in the previous utterance associated with the intent.
 4. Themethod of claim 1 wherein the stateless API request stores a profile ofthe user.
 5. The method of claim 1 further comprising training thesequential neural network prior to the sequential neural networkreceiving the data input, the training comprising feeding an untrainedalgorithm with multiple sequences of labels, each label stored in eachsequence of labels being associated with a sentiment score, wherein thetraining modifies the untrained algorithm to become the trainedalgorithm.
 6. The method of claim 1 wherein: the interactive responsesystem is an interactive voice response system; the user utterance is averbal message expressed by the user to an interactive voice responsesystem; and the transmitting the response to the user comprises theinteractive voice response system generating an audio message to theuser.
 7. The method of claim 1 wherein: the interactive response systemis a chatbot; the user utterance is a text message, or an electronicselection of a selectable icon, input by the user into a graphical userinterface displayed on a user mobile device; and the transmitting theresponse to the user comprises the chatbot generating a text message, ordisplaying a selectable icon, on the graphical user interface.
 8. Themethod of claim 1 wherein: the interactive response system is acloud-based API; and the interactive response system generates thestateless API request.
 9. The method of claim 1 further comprisingtransmitting the label associated with the first rule to the sequentialneural network classifier.
 10. A method for providing pre-processing ofan utterance prior to feeding utterance-related data to a sequentialneural network classifier for conversation sentiment scoring, theutterance being expressed, by a user, to an interactive response systemduring an interaction between the user and the interactive responsesystem, the method comprising: a conversation manager receiving arequest including the utterance, previous utterance data and a sequenceof labels, each label being associated with a previous utteranceexpressed by a user during the interaction; a natural language processorprocessing the utterance to output an utterance intent; a signalextractor:  processing the utterance, the utterance intent, the semanticmeaning, the utterance parameter, and the previous utterance data togenerate one or more utterance signals, wherein the one or moreutterance signals corresponds to a most probable intent of the user; and when the one or more utterance signals has been previously generatedduring the interaction and has previously not satisfied the user, thenmodifying the one or more utterance signals to correspond to a next mostprobable intent of the user; an utterance sentiment classifier:  storinga hierarchy of rules, each rule in the hierarchy of rules beingassociated with one or more rule signals and a label;  in response toreceiving the one or more utterance signals from the signal extractor,iterating through the hierarchy of rules in sequential order to identifya first rule in the hierarchy for which the one or more utterancesignals are a superset of the rule's one or more rule signals; thesequential neural network classifier:  receiving a data input includingthe sequence of labels and a label associated with a rule identified bythe utterance sentiment classifier, the data input not including theutterance;  processing the data input using a trained algorithm; and outputting a sentiment score; the conversation manager:  identifying aresponse to the user utterance based on the utterance intent, the labeland the sentiment score; and  transmitting the response to theinteractive response system; and the interactive response systemreceiving the response and outputting the response to the user; wherein: the pre-processing of the utterance by the natural language processor,the signal extractor and the utterance sentiment classifier reduces thedata input to the label and the sequence of labels, thereby increasing aspeed at which the sequential neural network classifier returns thesentiment score and decreasing resources consumed by the sequentialneural network classifier when processing the data input.
 11. The methodof claim 10 further comprising transmitting the label associated withthe first rule to the sequential neural network classifier.
 12. Themethod of claim 11 wherein transmitting of the response to theinteractive response system includes the conversation manager:augmenting the request to include the response; and transmitting theaugmented request to the interactive response system.
 13. The method ofclaim 10 wherein the request is a stateless API request.
 14. The methodof claim 13 wherein: the interactive response system is a cloud-basedAPI; and the interactive response system generates the stateless APIrequest.
 15. Apparatus for providing pre-processing of an utteranceprior to feeding utterance-related data to a sequential neural networkclassifier for conversation sentiment scoring, the utterance beingexpressed, by a user, to an interactive response system during aninteraction between the user and the interactive response system, theapparatus comprising: a conversation manager comprising a firstprocessor for receiving a stateless application programming interface(“API”) request including the utterance, previous utterance data and asequence of labels, each label being associated with a previousutterance expressed by a user during the interaction; a natural languageprocessor for processing the utterance to output an utterance intent, asemantic meaning of the utterance and an utterance parameter, theutterance parameter comprising one or more words included in theutterance and being associated with the intent; a signal extractor for: processing the utterance, the utterance intent, the semantic meaning,the utterance parameter, and the previous utterance data to generate oneor more utterance signals, wherein the one or more utterance signalscorresponds to a most probable intent of the user; and  when the one ormore utterance signals has been previously generated during theinteraction and has previously not satisfied the user, then modifyingthe one or more utterance signals to correspond to a next most probableintent of the user; an utterance sentiment classifier comprising:  amemory for storing a hierarchy of rules, each rule in the hierarchy ofrules being associated with one or more rule signals and a label;  asecond processor for: in response to receiving the one or more utterancesignals from the signal extractor, iterating through the hierarchy ofrules in sequential order to identify a first rule in the hierarchy forwhich the one or more utterance signals are a superset of the firstrule's one or more rule signals; the sequential neural networkclassifier for:  receiving a data input including the sequence of labelsand a label associated with the rule identified by the utterancesentiment classifier, the data input not including the utterance; processing the data input using a trained algorithm; and  outputting asentiment score; the conversation manager for:  identifying a responseto the user utterance based on the utterance intent, the label and thesentiment score;  augmenting the stateless API request to include theresponse, the utterance intent, the semantic meaning, the utteranceparameter, the label and the sentiment score; and  after the augmenting,transmitting the stateless API request to the interactive responsesystem; and the interactive response system for receiving the statelessAPI request and outputting the response to the user; wherein:  thepre-processing of the utterance by the natural language processor, thesignal extractor and the utterance sentiment classifier reduces the datainput to the label and the sequence of labels, thereby increasing aspeed at which the sequential neural network classifier returns thesentiment score and decreasing resources consumed by the sequentialneural network classifier when processing the data input.
 16. Theapparatus of claim 15 wherein the utterance sentiment classifier isfurther configured to transmit the label associated with the first ruleto the sequential neural network classifier.
 17. The apparatus of claim15 wherein the previous utterance data includes, for each previousutterance expressed by the user during the interaction: the previousutterance; an intent of the previous utterance; and a parameter of theprevious utterance.
 18. The apparatus of claim 15 wherein the previousutterance data includes data relating to a predetermined number ofprevious utterances immediately preceding the utterance, the previousutterance data, for each previous utterance, including: the previousutterance; an intent of the previous utterances; and a term in theprevious utterance associated with the intent.
 19. The apparatus ofclaim 15 further comprising a third processor for feeding the sequentialneural network with training data prior to the sequential neural networkreceiving the data input, the training comprising feeding an untrainedalgorithm with multiple sequences of labels, each label stored in eachsequence of labels being associated with a sentiment score, wherein thetraining modifies the untrained algorithm to become the trainedalgorithm.